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[57] ABSTRACT 

In an image processing system utilized with a car navigation 
system, an image photographed by a digital camera is 
coordinated with a map image in relation to this photo- 
graphed image, so that a driver can readily confirm where 
this photographed image has been acquired with respect to 
the map. The image processing system includes: an image 
storage unit for storing photographed image data and posi- 
tional information data acquired in correspondence with the 
photographed image data; a map image storage unit for 
storing map image data; a retrieval unit for retrieving a 
position with respect to the map image data stored in the 
map image storage unit based upon the positional informa- 
tion data stored in the image storage unit; and an image 
output unit for outputting a map image of the map image 
data about the position retrieved by the retrieval unit, and a 
photographed image of the photographed image data in 
correspondence with the map image data. 

2 Claims, 16 Drawing Sheets 
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IMAGE PROCESSING SYSTEM CAPABLE 
OF DISPLAYING PHOTOGRAPHED IMAGE 
IN COMBINATION WITH RELEVANT MAP 
IMAGE 

TECHNICAL FIELD 

The present invention generally relates to an image pro- 
cessing system and an image processing method. More 
specifically, the present invention is directed to an image 
processing method and system capable of processing image 
data acquired by an electronic camera and map data to 
display a photographed image in relation with a map. 

BACKGROUND ART 

Very recently, so called "digital cameras" (namely, elec- 
tronic cameras) are commercially available. In a digital 
camera, an image of a subject optically received by a CCD 
image sensor corresponding to a solid-state imaging system 
is acquired as a digital image signal, and then this digital 
image signal is saved in a RAM. Since the image of the 
photographed subject is converted into the digital image 
data, this digital image data can be transferred/processed in 
digital information processing apparatuses such as personal 
computers and wordprocessors, so that digital cameras may 
function as easy-operable image input devices having wide 
applications. 

However, this sort of image input device merely saves 
photographed images. Therefore, in order to grasp where the 
image is acquired, a photographer must make a place name 
memorandum when this image is taken. Then, when this 
photographed image data is transferred to the 
wordprocessor, the photographer should enter the photo- 
graph place by manipulating the keyboard with reference to 
this memorandum. 

On the other hand, car navigation systems are commer- 
cially available, in which present positions are sequentially 
detected by utilizing satellite communications, and map 
image data about the detected positions are displayed in real 
time in accordance with this position detection. In this sort 
of car navigation, although the present place may corre- 
spond to the map image, merely the map is displayed or a 
voice response is made. This type of car navigation system 
cannot supply positional information with respect to photo- 
graphs also cannot provide a navigation function with 
respect to photographs. 

DISCLOSURE OF THE INVENTION 

The present invention has been made to solve the above- 
described problems, and therefore has an object to provide 
an image processing system, and an image processing 
method which are capable of easily recognizing a photo- 
graphing location by outputting photographed image data 
and map image data indicative of this photograph place in 
relation thereto. Moreover, another object of the present 
invention is to provide an image processing system and an 
image processing method which are capable of effectively 
achieving a navigation function by outputting map image 
data corresponding to a present position, and further out- 
putting previously photographed image data about this posi- 
tion when a vehicle comes to this position. 

To achieve the above-described objects, an image pro- 
cessing system, according to a first aspect of the present 
invention, is featured by comprising: 

image storage means for storing photographed image data 
and positional information data acquired in correspon- 
dence with said photographed image data; 



3,947 

2 

map image storage means for storing map image data; 
retrieval means for retrieving a position with respect to 
the map image data stored in said map image storage 
means based upon the positional information data 
5 stored in said image storage means; and 

image output means for outputting a map image of the 
map image data about the position retrieved by said 
retrieval means, and a photographed image of the 
photographed image data in correspondence with said 
10 map image data. 

Also, an image processing system, according to a second 
aspect of the present invention, is featured by comprising: 
image record means for recording data about a photo- 
graphed image; 

15 present position storage means for storing data about 
present positional information of a photograph place in 
correspondence with said photographed image data; 
map image storage means for storing data about a map 
image; 

20 retrieval means for retrieving a position with respect to 
said map image data based upon said present positional 
information data stored in said present position storage 
means; and 

image output means for outputting the map image data 
25 retrieved by said retrieve means, and the photographed 
image data in correspondence with said image data. 
In accordance with a third aspect of the present invention, 
an image processing system coupled to a navigation system 
for sequentially detecting a present position and for sequen- 
30 tially outputting map image data about said present position 
in response to the detected present position, is featured by 
comprising: 

photograph means for optically receiving an image of a 
photographed subject by a solid-state imaging element 
35 to produce digital image data; 

image storage means for storing the digital image data 

produced from said photograph means; 
present position storage means for storing present posi- 
tional information data about a place photographed by 
40 said photograph means in correspondence with said 
photographed image data as photographed present 
positional information data; 
comparing means for comparing said photographed 
present positional information data stored in said 
45 present position storage means with said present posi- 
tion data produced by the navigation system; and 
image output means for reading out the photographed 
image data corresponding to said photographed present 
5Q position information based upon a comparison result of 
said comparing means to thereby output the image of 
the photographed subject. 
Also, an image processing method, according to a fourth 
aspect of the present invention, is featured by comprising the 
55 steps of: 

photographing an image of a subject to produce photo- 
graphed image data; 

acquiring positional information in correspondence with 
said photographed image data to store the acquired 
60 positional information; 

previously storing map image data; 

retrieving a position with respective to the stored map 
image data based on the acquired positional informa- 
tion; and 

65 outputting a map image of map image data about said 
retrieved position, and a photograph image of said 
photographed image data in relation thereto. 
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Further, according to a fifth aspect of the present 51 for constituting an image processing system according to 

invention, a storage medium is featured by comprising at a third embodiment of the present invention; 
least: FIG. 15 is a flow chart for describing operations of the 

a program code readable by a computer, for retrieving a GPS on-vehicle terminal unit 51 in an image recording 

position with respect to stored map image data based 5 process; 

upon photographed positional information stored in FIG. 16 is a flow chart for representing operations of the 

correspondence with photographed image data derived GPS on-vehicle terminal unit 51 in a normal mode; and 
from a photographing unit; and FIGS. 17A to 17E explanatorily represent display image 

another program code readable by the computer, for samples of the GPS on-vehicle 51 in the normal mode and 

outputting the retrieved map image data and the pho- 10 an image recording mode. 

tographed image data in relation therewith. BEST MODE OF CARRYING OUT THE 

Moreover, according to a sixth embodiment of the present INVENTION 

invention, a storage medium is featured by comprising at „ „ . , . . 

2 east . ° Referring now to the drawings, a description will be made 

, , , , c 15 of various image processing systems according to presently 
a program code readable by a computer, for comparing preferred embodiments of the present invention, 
photographed positional information stored in corre- [Arrangement of First Digital camera] 
spondence with photographed image data derived from FIG. 1 is a schematic block diagram for showing an 
a photographing unit with present positional informa- ovefall of a so<alled camera (namd 
tion acquired by a navigation system; and 2Q electronic camera) » which may consti t u te an image process- 
another program code readable by the computer, for m g system according to a first preferred embodiment of the 
reading out the photographed image data correspond- present invention. The image processing system includes the 
ing to said photographed positional information based digital camera 1 and a wordprocessor 21 functioning as an 
on a comparison result to thereby output said photo- external appliance electrically connected to this digital cam- 
graphed image data. 25 era 1 by way of an electric cable. 

The digital camera 1 is a portable type electronic still 

BRIEF DESCRIPTION OF THE DRAWINGS camera oper able by a cell. This digital camera 1 is arranged 

A more complete understanding of the teachings of the bv > ™ shown in nG - a CPU 2 » a stora S e unit 3 > a CCD 

present invention may be acquired by referring to the ™ a g e in P ut umt 4 > aQ Paging lens 5, an image sensor I/F 

accompanying figures, in which like reference numbers 30 circuit 6 > a dis P la y contro1 unit 7 > a dis P la y unit 8 > a ke X 

indicate like features and wherein: operation unit 9, a RAM 10, and an external data input/ 

™„ - . , *• ui i j- r *• output unit 11. The CPU (central processing unit) 2 is such 

FIG. 1 is a schematic block diagram for representing an r , , . L / - 4 «• 

„ . r j * ** i i r *• a central processing unit capable of controlling an entire 

overall arrangement of a digital camera 1 for constituting an r -.. t . r i . 4 & , ^ 

& . 4 " t - , ,j. ~ - operation of this digital camera lin response to vanous sorts 

image processing system according to a first embodiment of £ . p . r t _ 

iL & r . & y & 35 of programs previously stored in the storage unit 3. The 

the present invention. * ■* 4 * - * j- r i 

v storage unit 3 contains a storage medium for previously 

FIG. 2 explanatorily shows an outer view of the digital storing taer e mt o a program and data. This storage medium 

camera 1 indicated in FIG. 1; may be made from a magnetic storage medium, an optical 

FIG. 3 represents a structure of the key operation unit 9 storage medium, or a semiconductor memory. The storage 

in FIG. 1; 40 medium may be fixedly employed in the storage unit 3, or 

FIG. 4 schematically indicates a storage area structure of may be detachably provided with the storage unit 3. 

the RAM 10 shown in FIG. 1 and a data structure thereof; Alternatively, the above-described program and data may be 

FIG. 5 is a schematic block diagram for representing an accepted from other appliances via a communication fine to 

internal arrangement of the wordprocessor 21 connectable to be stored int0 this storage medium. Furthermore, another 

the digital camera 1 of FIG. 1; 45 stora S e ^ equipped with the storage medium is provided 

, . n , t r , ■ ■ c tU on the side of other appliances connected thereto via a 

FIG. 6 is a flow chart for exp laming operations of the . . r *\ u , 4 , , . 

. + . . t • * communication line, and the program and/or data stored in 

digital camera 1 in a photograph mode; . . . * . , . . . 

& r & r mis storage medium may be used via this communication 

FIGS. 7A to 7E explanatorily show key operation/image Uae CCD image input unit 4 fe a CCD ( c h a rge-coupled 

sample of the digital camera 1 in the photograph mode; 50 device) i mage sensor as a solid-state imaging element. An 

FIG. 8 is a flow chart for describing operations of the image of a photographic object which is received via the 

digital camera 1 in a reproduce mode; imaging lens 5 is photoelectrical^ converted into an image 

FIGS. 9A to 9D explanatorily indicate operation/image signal by the CCD image input unit 4. Then, the image 

sample of the digital camera 1 in the reproduce mode; signal is supplied to the image sensor I/F circuit 6. The 

RG. 10 is a flow chart for explaining operations of the 55 ima g e I/F circuit 6 converts the analog image signal 

wordprocessor 21 in an image synthesizing process; mt0 a dl S ital una e e sl S naL This digitally photographed 

itjpp ii a » i-i n i * i • j* * • i image data is fetched into the CPU 2. In this embodiment, 

FIGS. UA to UD explanatorily indicate an image sample >.t«t ^ i**t.i_ t 

riU j ~t . . ' . . . b r the CPU 2 supplies the photographed image data as one 

of the wordprocessor 21 in the image synthesizing process; . f \ . 4 . K. , ° r » 

r & J & r image screen data to the display control unit 7. One image 

FIG. 12 is a schematic block diagram for showing an 60 screen data is fon^ed t0 the display unit 8 for display 

overall arrangement of a GPS on-vehicle terminal unit 41 for purposes. The display unit 8 is constructed of a liquid crystal 

constituting an image processing system according to a display device and me lihe> and may mnction ^ ^ 

second embodiment of the present invention; reproduced screen or a finder screen. When a shutter switch 

FIG. 13 explanatorily illustrates an outer view of the GPS ( no t shown in detail) employed on the key operation unit 9 

on-vehicle terminal unit 41 shown in FIG. 12; 65 js operated in this digital camera 1, the image data obtained 

FIG. 14 is a schematic block diagram for indicating an during this photographing operation is stored into the RAM 

overall arrangement of another GPS on-vehicle terminal unit 10 as one screen image data. 
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The RAM (random access memory) 10 is an image 
memory for temporarily storing thereinto the photographed 
image data and various sorts of input information. When an 
image reproducing instruction is issued from the key opera- 
tion unit 9, the CPU 2 reads both the image data instructed 
for the image reproduction, and the input information cor- 
responding thereto from the RAM 10, and then supplies this 
image data together with the corresponding input informa- 
tion to the display control unit 7 so as to display the image 
on the display unit 8. Also, when an image transferring 
instruction is entered from the key operation unit 9, the CPU 
2 transfers the photographed image data stored in the RAM 
10 via the external data input/output unit 11 to the word- 
processor 21. It should be noted that this external data 
input/output unit 11 corresponds to, for example, an RS232C 
serial interface. It should also be understood that although 
the wordprocessor 21 is employed as the external appliance 
of this first image processing system, when a floppy disk unit 
is employed as this external appliance, the external data 
input/output unit 11 corresponds to a disk driver. 
[Outer View of First Digital Camera] 

FIG. 2 shows an outer view of the digital camera 1 of FIG. 
1. On the side of a front face, the imaging lens 5 and the 
CCD image input unit 4 positioned opposite to this imaging 
lens 5 are built in this digital camera 1. Various sorts of keys 
(will be discussed later) for constructing the key operation 
unit 9 are provided in an upper face. A cord connection 
terminal (jack) 12 for connecting the external data input/ 
output unit 11 to the wordprocessor 21 in a side face. Then, 
the display unit 8 is provided in a rear face of the digital 
camera 1, and may function as a finder screen under such a 
condition that the imaging lens 5 is directed to the photo- 
graph object. 

[Panel of Key Operation Unit] 

FIG. 3 is a panel for representing the key operation unit 

9 in more detail. This key operation unit 9 is provided with 
a power switch "POWER", a shutter switch "SHOT", a 
slide-type mode selecting switch "MOD" for selecting a 
photograph mode (PLAY)/a reproduce mode (REC), and 
various sorts of keys Kl, K2, K3, K4, K5. These keys own 
the following functions. For instance, in the case that the 
input information is set to the RAM 10 in correspondence 
with the photographed image, the OK key Kl may function 
as an execution key and a carry key, whereas the NG key K2 
may function as a release key and a shift down key. Every 
time the plus key K3 is manipulated, the numeral data is 
incremented by "1", namely a numeral value entry key. 
Every time the minus key K4 is manipulated, the numeral 
data is decremented by "1", namely a numeral value entry 
key. In the reproduce mode, the plus key K3/the minus key 
K4 may function as image selection keys. In this reproduce 
mode, the functioning key corresponds to a key used to 
instruct whether or not the input information set in the RAM 

10 in correspondence with the image to be produced is 
displayed in combination with the photographed image. 
[Storage Contents of RAM 10] 

The storage contents of the RAM 10 will now be 
described with reference to FIG. 4. 

As represented in FIG. 4, the RAM 10 contains a system 
work area 10-1 used to store various sorts of conditions, an 
image data area 10-2 used to store a photographed image and 
input information corresponding to this photographed 
image, a stack area 10-3 for temporarily storing data. Now, 
the storage content of the image data area 10-2 will be 
explained. For instance, a first image is employed, and a data 
format thereof is indicated. That is, data is arranged by 
identifiers for data byte numbers+data sorts (identifiers for 
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identifying positional information, additional information, 
image information, and control information), and actual 
data. In this embodiment, a first record, namely a record 
containing positional information as an identifier is consti- 

5 tuted by a flag indicative of north/south/east/west (1 byte), 
latitude data (5 bytes), longitude data (5 bytes), and an 
identifier (1 byte), i.e., a total of 12 bytes. This actual 
positional information data is set by entering from the key 
operation unit 9, present position information being dis- 

10 played on a display screen of a car navigation system. Then, 
a data length (12 bytes), the identifier (positional 
information), the actual position information data (latitude/ 
longitude data etc.) are set to the RAM 10 in the arranging 
order. 

15 The next record, namely a record containing additional 
information as an identifier, is arranged by a data length (22 
bytes), an identifier (additional information), and actual 
additional information data (21 bytes) in this order. The 
actual additional information data corresponds to, for 

20 example, a title of an image. Furthermore, a record contain- 
ing image information as an identifier is stored. Since the 
actual image information data requires a large memory 
capacity, a data length is indicated by a quantity of block, 
while 256 bytes are handled as 1 block. FIG. 4 concretely 

25 represents the contents of the actual image information data 
in unit of byte. That is, the actual image information data are 
subdivided into three primary colors such as red, blue, green 
colors every 1 dot from the upper left 1 dot of the image, 
which are stored. In this case, as indicated in FIG. 4, the 

30 image is arranged by 240 dotsx320 dots. As a result, 
approximately, 230 Kbytes defined by 240 dotsx320 dotsx3 
(byte number for three primary colors) may constitute the 
actual image information data. It should be noted that as to 
the final record, an end code indicative of an end of the first 

35 data is used as actual data thereof. The CPU 2 may store the 
photographed image data into the RAM 10 in such a fifing 
format. 

[Internal Arrangement if Wordprocessor] 

Referring now to a block diagram of FIG. 5, an internal 
40 arrangement of the above-described wordprocessor 21 will 
be described. 

This wordprocessor 21 is mainly arranged by the below- 
mentioned units/circuit elements. That is, a CPU 22 is 
employed so as to control the overall operation of this 

45 wordprocessor 21 in accordance with various sorts of pro- 
grams stored in a storage unit 23. When document data is 
inputted from a key operation unit 24, the CPU 22 converts 
this entered document data into display data, and then 
supplies this display data to a display control unit 25 so as 

50 to display the document on a display unit 26 based on this 
display data. Furthermore, the CPU 22 stores document data 
defined by kana/kanji-character conversions into a RAM 27. 
The storage unit 23 contains a storage medium for previ- 
ously storing thereinto a program and data. This storage 

55 medium may be made from a magnetic storage medium, an 
optical storage medium, or a semiconductor memory. The 
storage medium may be fixedly employed in the storage unit 
23, or may be detachably provided with the storage unit 23. 
Alternatively, the above-described program and data may be 

60 accepted from other appliances via a communication line to 
be stored into this storage medium. Moreover, another 
storage unit equipped with the above-explained storage 
medium is provided which is connected via a communica- 
tion line to other appliances, and then the program and the 

65 data previously stored in this recording medium may be used 
through the communication line. Also, various sorts of 
interface circuits are connected to the CPU 2. Namely, a disk 
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I/F circuit 28, a printer I/F circuit 29, a player I/F circuit 30, 
and a camera I/F circuit 31. Thus, this CPU 22 may control 
input/output operations of a floppy disk drive 32, a printer 
unit 33, a CD-ROM player 34, and the digital camera 1 via 
these I/F circuits 28 to 31. 5 

The CD-ROM player 34 drives a CD-ROM 35 used in a 
car navigation system. This CD-ROM 35 is constituted by 
storing thereinto positional information of maps in corre- 
spondence with various sorts of map image data. The camera 
I/F circuit 31 is an RS-232 serial interface circuit. The CPU 
22 acquired photographed image data transferred from the 
digital camera 1 and positional information indicative of this 
photographing place via the camera I/F circuit 31. In this 
case, the CPU 22 retrieves the map image data stored in the 
CD-ROM 35 based upon the positional information acquired 
from the digital camera 1. Then, the CPU 22 acquires this 15 
map image data, and combines or synthesizes this map 
image data with the photographed image data to produce 
synthesized image data. The synthesized image data is 
stored in the RAM 27. In this case, the synthesized image 
data is stored in such a manner that this synthesized image 20 
data is inserted into the document data. The memory con- 
tents of the RAM 27 are recorded, or registered via the disk 
I/F circuit 28, and the floppy disk drive 32 on the floppy 
disk. Alternatively, instead of the floppy disk, an optical 
memory disc having a large memory capacity and/or a 25 
high-speed memory card may be employed so as to record/ 
save the memory contents of the RAM 27. 
[Photograph Mode Operation of Digital Camera] 

FIG. 6 is a flow chart for representing operations of the 
digital camera 1 in a photograph mode. 30 

It should be-understood that a program used to realize 
various functions described in this flow chart has previously 
been stored in the above-explained storage medium in a 
program code format readable by the CPU (this is similarly 
applied to another flow chart of FIG. 8). 35 

That is, when the mode selecting switch MOD is changed 
into the REC position, the CPU 2 executes the operations in 
accordance with the process sequence of the flow chart 
shown in FIG. 6. FIGS. 7A to 7E schematically represent 
key operations and image samples in the photograph mode. 40 

First, in the flow chart of FIG. 6, the CPU 2 causes the 
CCD image input unit 4 and the image sensor I/F circuit 6 
to be driven. Thereafter, the image of the photographed 
object is optically received by the CCD image input unit 4 
so as to be photoelectrically converted into the analog image 45 
signal. Then, this analog image signal is converted into the 
digital image data by the image sensor I/F circuit 6. The 
CPU 2 acquires this photographed image data (step Al) and 
displays the photographed image on the display unit 8 (step 
A2). Then, since the process operations defined at the steps 50 
Al and A2 are repeated until the shutter switch SHOT is 
manipulated, the photographer operates the digital camera 1 
to be focused onto a desired scene while confirming the 
displayed image functioning as the finder. When the shutter 
switch SHOT is depressed, the CPU 2 stores the photo- 55 
graphed image data acquired at this time into the RAM 10, 
and stops the image acquisition (step A4). Now, FIG. 7A 
represents a photographed image produced when the shutter 
switch SHOT is depressed. Under such a condition that the 
photographed image is displayed, the CPU 2 displays mes- 60 
sage data for inquiring whether or not the photographing 
place is recorded in relation to this photographed image on 
the display screen of the display unit 8 (step A5). FIG. 7B 
represents a display condition in this case. When the pho- 
tographer confirms the content of this displayed message 65 
and wishes to record the photographing place, the photog- 
rapher manipulates the OK key Kl (step A6). 



In such a case that the positional information about the 
photographing place, the photographer selectively manipu- 
lates the OK key Kl, the NG key K2, the plus key K3, and 
the minus key K4, while confirming the present position 
being outputted on the display screen of the car navigation 
system, so that the numeral values of the latitudinal/ 
longitudinal data are inputted one digit by one digit (step 
A7). In other words, when the plus key K3 is manipulated, 
the numeral value is incremented, whereas when the minus 
key K4 is manipulated, the numeral value is decremented. 
When, the OK key Kl is manipulated, the present digit is 
advanced to the subsequent digit, whereas when the NG key 
K2 is manipulated, the present digit is returned to the 
preceding digit. Accordingly, these keys are selectively 
manipulated, so that the positional information can be 
entered (see FIG. 7C). As a result, when all of the desired 
digits have been entered, as shown in FIG. 7D, another 
message is displayed in order to finally confirm as to 
whether or not the entered positional information is defined 
(step A8). If the entered positional information is defined, 
then the OK key Kl is manipulated (step A9). Accordingly, 
the CPU 2 shifts the memory content of the RAM 10 by a 
byte amount equal to the byte number of the entered 
positional information data along the backward direction, 
and then stores the positional information data into the RAM 
10 (step A10). Also, the CPU 2 stores the photographed 
image data into this RAM 10 in relation to the present 
positional information. Then, the CPU 2 displays such a 
message that the positional information about the photo- 
graph place has been recorded under normal condition, and 
increments a pointer (not shown) for designating a write 
address of the RAM 10 (step All). This display condition is 
shown in FIG. 7E. Then, the process operation of the 
photograph mode is returned to the previous step Al, and 
then a similar process operation is repeated. 
[Reproduction Mode Operation of Digital Camera] 

Referring now to another flow chart shown in FIG. 8, 
operations of the digital camera 1 will be described in the 
reproduce mode when the mode selection switch MOD is 
changed into a PLAY position. FIGS. 9A to 9D represent key 
operations/image samples during the reproduce mode. 

When the operation mode is switched to the reproduce 
mode, the CPU 2 first stops the drive operations of the CCD 
image input unit 4 and the image sensor I/F circuit 6, and 
thereafter reads out a top image (namely, an image stored at 
a top storage area) of the RAM 10 to thereby display this top 
image (step Bl). In this case, a check is done as to whether 
or not the positional information indicative of the photo- 
graph position is contained in this top image data (step B2). 
If the positional information is contained in this top image 
data, then another check is made as to whether or not a 
position display flag is turned ON (step B3). Although the 
function of this position display flag will be described more 
in detail, the position display flag is turned ON when such 
an instruction is made that the positional information is 
previously displayed. Since the position display flag is 
turned OFF at a first stage, the process operation in the 
reproduce mode is advanced to a step B5 at which the CPU 
2 waits for a key entry. 

Under this key entry waiting condition, the plus/minus 
keys K3/K4 may be manipulated (step B6), the function key 
may be operated (step B8), and the OK key Kl may be 
operated (step B10). It is now assumed that as indicated in 
FIG. 9A, the function key K5 is manipulated under such a 
condition that the top image is reproduced (step B8). As a 
result, the process operation is advanced to a next step B9 at 
which a guidance indication for inquiring whether or not the 
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photograph place is displayed, as shown in FIG. 9B. In this the positional information is added to the selected image, the 

case, when the OK key Kl is manipulated so as to instruct CD-ROM 35 is set and a guidance for instructing the 

the display of the photograph position (step BIO), the manipulation of the execution key is displayed (step C8). 

above-described position indication flag is turned ON (step This condition is displayed in FIG. 1C. Now, when the 

Bll). Conversely, when the OK key Kl is not operated, the 5 CD-ROM 35 used in the car navigation system is set, the 

turn-OFF state of the position display flag still remains (step CPU 12 retrieves the CD-ROM 35 based on the positional 

B12). Under such a condition that the position information information of the photographed image (step C9), and reads 

has teen added to the presently reproduced image (step B2) &Qm the CD _ R0M 35 such map image data defined in a 

and furthermore the position display flag is turned ON (step lected from , - itkm coincident 

B3) so as to return to the image display process (step Bl), , n tU i ■ r *• r *u- u * u j • 

an window for displaying the positional information is 10 ™ th * e J^T^i ^™^ n ^ ^ P toto ^P^ m W 

opened within this displayed image, and also the positional ^ te ? CIO) This CPU 22 stores this read map image data into 

information is displayed as indicated in FIG. 9C. As the image file and also stores an upper left coordinate pomt 

described above, the positional information is displayed of this map image in correspondence with this photographed 

under such an initial conditional that the function key K5 ima S e ( ste P C11 >- Subsequently, when an arrangement 

and the OK key Kl are manipulated. This is because if the 15 w^ 11 a document is designated (step C12). When this 

positional information is continuously displayed, the image arrangement designation is carried out by designating a 

display area will be narrowed. position, a size, and a title of an image file, the map image 

Now, when either the plus key K3 or the minus key K4 is data is arranged within the document which has been formed 

manipulated so as to selectively display a desired image by the wordprocessor 21 in accordance with this designation 

(step B6), the CPU 2 moves the RAM pointer to the next 20 (step C13). 

image when the plus key K3 is manipulated, and moves the Next, a calculation process is carried out so as to calculate 

RAM pointer to the preceding image when the minus key K4 an offset position for defining where the photographed 

is operated (step B7). Then, since the process operation is image is displayed at a position on the map (step C14). 

returned to the image display process (step Bl), the images Assuming now that the upper left coordinate point of such 

are switched every time the plus key K3 and/or the minus 25 a map image as shown in FIG. 10 is stored, a check is done 

key K4 is manipulated. As a consequence, when a desirable as to whether or not there is a place corresponding to the 

image is selected, if the positional information is contained position of the photographed image at the position shifted 

in this desirable image, then the selected desirable image is from this upper left coordinate point as the reference point, 

displayed together with this positional information, as indi- Thereafter, a "ballon pattern" is formed. This ballon pattern 

cated in FIG. 9D, under such a condition that the position 30 is arranged in such a manner that a tip portion of this ballon 

display flag is turned ON (step B3). pattern is located at the photograph place within the map, 

[Image Synthesizing Process by Wordprocessor] and the positional information of the photographed image is 

FIG. 10 is a flow chart for indicating an image synthe- displayed within this spout pattern (step C15). The image is 

sizing process operation by the wordprocessor 21 shown in arranged in such a manner that the photographed image may 

FIG. 1. 35 be stored within this spout pattern (step C16). FIG. 11D 

A program for realizing various functions described in represents an image sample produced by synthesizing the 

this flow chart is previously stored in the above-described map image and the photographed image within the docu- 

storage medium in a program code format readable by the ment. In this case, when a document is edited, a character 

CPU 22 of FIG. 5. FIGS. 11 A to 11 D show key operations/ series about a photograph place and a photographer memory 

image samples in this image synthesizing process operation. 40 may be entered into this document by manipulating the keys. 

First, when an image synthesizing mode is designated, the In the image processing system with the above-described 

CPU 22 displays such a guidance for connecting the digital arrangement according to the first embodiment, the posi- 

camera 1 with this wordprocessor 21 (see FIG. 11A). As a tional information indicative of this photograph place is 

result, when the digital camera 1 is connected to the camera entered into the image photographed by the digital camera 1 

I/F circuit 31 (step CI), the image data is sequentially 45 by manipulating the keys in relation thereto. As a result, 

transferred from the digital camera 1 (step C2), and thus a when the photographed image is inserted into the document 

desired image is selected by way of the key operation (step in the wordprocessor 21, the map image corresponding to 

C3). As illustrated in FIG. 11B, this image selection is this photograph place is read out from the CD-ROM 35 used 

performed by operating the execution key in such a manner in the car navigation system, and the synthesized image 

that the present image is returned to the preceding image by 50 produced by synthesizing the photographed image with this 

manipulating the preceding subject key, and/or the present map image is inserted into the document. Accordingly, there 

image is advanced to the succeeding image by manipulating are advantages that the photograph place can be visually 

the succeeding subject key. As described above, the selec- recognized and also can be effectively displayed, 

lively determined image data is stored in the image file, so It should be understood that although the synthesized 

that even when the digital camera 1 is not connected to this 55 image produced by synthesizing the map image with the 

wordprocessor 21 during the subsequent operation, the photographed image is inserted into the document, only the 

photographed image data may be arbitrarily read out so as to synthesized image may be alternatively output ted. 

be displayed and/or printed out (step C4). Next, a check is Moreover, the map image is not synthesized with the pho- 

done as to whether or not the arbitrary information is added tographed image, but may be separately outputted for dis- 

to this image (step C5). If no arbitrary information is 60 play purposes. 

contained in this image, then the process operation is [Arrangement/Operation of Second Image Processing 

advanced to a process for simply arranging the images (step System] 

C6). To the contrary, if the arbitrary information is contained An image processing system according to a second 

in this image, then the process operation is advanced to step embodiment of the present invention will now be explained 

C7 to step C16. 65 with reference to FIG. 12 and FIG. 13. 

First, the CPU 22 extracts the positional information in It should be understood that although the digital camera is 

relation to the photographed image (step C7). Then, since separately provided with the CD-ROM in the above- 
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described first embodiment, a digital type electronic still 
camera is assembled in an on-vehicle terminal unit of a car 
navigation system in a second embodiment of the present 
invention. That is, according to the image processing system 
of the second embodiment, positional information about a 
photograph place is not entered by manipulating the keys, 
but may be automatically inputted. 

FIG. 12 is a block diagram for showing a digital camera 
integrated type GPS on-vehicle terminal unit 41. It should be 
noted that the same reference numerals shown in the digital 
camera 1 of FIG. 1 and the wordprocessor 21 of FIG. 5 will 
be employed as those for denoting the same or similar 
elements/units indicated in FIG. 12, and therefore, no expla- 
nations thereof are further made. 

This GPS on-vehicle terminal unit 41 equipped with the 
digital camera is detachably mounted on a sun visor of a 
vehicle. A CPU (central processing unit) 42 is such a central 
processing unit capable of controlling an entire operation of 
this GPS on-vehicle terminal unit 41 in response to various 
sorts of programs previously stored in a storage unit 43. The 
storage unit 43 contains a storage medium for previously 
storing thereinto a program and data. This storage medium 
may be made from a magnetic storage medium, an optical 
storage medium, or a semiconductor memory. The storage 
medium may be fixedly employed in the storage unit 43, or 
may be detachably provided with the storage unit 43. 
Alternatively, the above-described program and data may be 
accepted from other appliances via a communication line to 
be stored into this storage medium. Furthermore, another 
storage unit equipped with the storage medium is provided 
on the side of other appliances connected thereto via a 
communication line, and the program and/or data stored in 
this storage medium may be used via this communication 
line. 

Similar to operations of the normal car navigation system, 
a positional information converting circuit 44 receives elec- 
tromagnetic wave signals transmitted by the satellite com- 
munication via an antenna 45 and converts the electromag- 
netic wave signals into a positional information signal. This 
positional information is acquired via a GPS I/F circuit 46 to 
the CPU 42. As a result, the CPU 42 may access the storage 
content of the CD-ROM 35 via a player I/F circuit 30 and a 
CD-ROM player 34, and thus causes the display unit 8 to 
display map image data corresponding to this positional 
information thereon. On the other hand, when the CPU 42 
acquires the images of the subject photographed/received 
via the imaging lens 5 to the CCD image input unit 4 via an 
image sensor I/F circuit 4 as a digital image, the CPU 42 
controls the display control unit 7 to display this subject 
image on the display unit 8, and also stores the digital image 
data into the RAM 10 in response to the operation of the 
shutter switch mounted on the key operation unit 9. 
Furthermore, positional information during this photograph- 
ing operation is acquired via the GPS I/F circuit 46, and then 
is stored in the RAM 10 in correspondence with the pho- 
tographed image. The CPU 42 transfers the storage contents 
of the RAM 10 via the external data input/output unit 11 to 
the wordprocessor 21. 

FIG. 13 is an outer view for representing the above- 
explained GPS on-vehicle terminal unit 41. Mounting hooks 
47 for mounting the GPS on-vehicle terminal unit 41 on the 
sun visor are provided on both right and left sides of an 
upper face. In FIG. 13, reference numeral 48 indicates an 
antenna storage unit, reference numeral 49 shows a key- 
board having character keys and the like. The keyboard 49 
is arranged in parallel to the display unit 8 on a rear surface 
of the GPS on-vehicle terminal unit 41. 
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In the GPS on-vehicle terminal unit 41 with the above - 
described arrangement, the positional information indicative 
of the photograph place can be automatically entered at the 
same time when the photographing operation is carried out, 

5 and can be stored into the RAM 10 in correspondence with 
the photographed image. In this case, since the GPS 
on-vehicle terminal unit 41 is mounted on the sun visor, the 
photographer can photograph the scene along the forward 
direction while driving the vehicle. It should be noted that 

10 similar to the above-described first embodiment, the 
memory contents of the RAM 10 may be transferred to the 
wordprocessor 21, and the synthesized image produced by 
synthesizing the photographed image with the map can be 
inserted into the document. 

is It should also be noted that although the GPS on-vehicle 
terminal unit 41 is mounted on the sun visor, the present 
invention is not limited thereto. That is, this GPS on-vehicle 
terminal unit 41 may be arbitrarily mounted on any places 
where the scenes can be photographed by the digital camera 

20 along the travel direction. 

[Arrangement of Third Image Processing System] 

An image processing system according to a third embodi- 
ment of the present invention will now be described with 
reference to FIG. 14 to FIGS. 17A-17E. As previously 

25 explained, the digital camera is assembled in the GPS 
on-vehicle terminal unit of the car navigation in the above- 
described second embodiment. In accordance with the third 
embodiment, a video camera for imaging a moving a picture 
is connectable with the on-vehicle terminal unit of the car 

30 navigation system. A still image is acquired from this video 
image and recorded in combination with positional infor- 
mation representative of this photograph (imaging) place. 
When a vehicle again comes to a place defined by this 
positional information, the still image related to this place is 

35 displayed, so that the navigation function can be improved. 
FIG. 14 is a block diagram for indicating an arrangement 
of a GPS on-vehicle terminal unit 51 employed in the third 
image processing system. It should be noted that the same 
reference numerals shown in the above-explained GPS 

40 on-vehicle terminal unit 4 of FIG. 12 will be employed as 
those for denoting the same or similar circuits/units of the 
GPS on-vehicle terminal unit 51 shown in FIG. 14. 

To this GPS on-vehicle terminal unit 51, a video camera 
61 is detachably connectable by way of a camera cable. The 

45 GPS on-vehicle terminal unit 51 is arranged by a video 
signal I/F circuit 52, a floppy disk driver 53, a disk I/F circuit 
54, and other circuit elements similar to those of the normal 
on-vehicle terminal unit. That is, a CPU (central processing 
unit) 55 is such a central processing unit capable of con- 

50 trolling an entire operation of this GPS on-vehicle terminal 
unit 51 in accordance with various sorts of programs pre- 
viously stored in a storage unit 56. The storage unit 56 
contains a storage medium for previously storing thereinto a 
program and data. This storage medium may be made from 

55 a magnetic storage medium, an optical storage medium, or 
a semiconductor memory. The storage medium may be 
fixedly employed in the storage unit 56, or may be detach- 
ably provided with the storage unit 56. Alternatively, the 
above-described program and data may be accepted from 

60 other appliances via a communication line to be stored into 
this storage medium. Furthermore, another storage unit 
equipped with the above-described storage medium is pro- 
vided on the side of other appliances connected via a 
communication line, and the program and data stored in this 

65 storage medium may be used via the communication line. 
Based upon present positional information derived from the 
GPS I/F circuit 46, map image data about this place is 
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acquired from the CD-ROM 35 via the player I/F circuit 30 [Normal Car Navigation Mode of GPS On- Vehicle Terminal 

to be displayed on the display unit 8. A key operation unit Unit 51] 

57 contains an image record key in addition to various sorts Referring now to a flow chart indicated in FIG. 16, 

of the normally employed keys. When an image recording operations of the GPS on-vehicle unit in the normal car 

instruction is issued by operating the key operation unit 57, 5 navigation mode will be described. 

the CPU 55 acquires video image data from the video p irst> positional information supplied from the positional 

camera 61 via the video signal I/F circuit 52, and tempo- information signal converting circuit 44 is converted into a 

rarily stores this video image data into the RAM 58 in di ital si ^ b lhe Gps w cifcuit 46 aod meQ mis 

combination with the present position al information, and s{ { fa ^ b lhe cpu 55 ( £1 ^ ^ M a 

further displays this video image. When an image saving consequence , me CPU 55 acquires a map image correspond- 

instruction is issued from the key operation unit 57, the CPU . \.. \ . . t , , . r / . , r .u 

55 supplies the memory content of the RAM 10 via the disk ^"J^ ! aformaU f ° n Slgnal fr ° m he 

I/F circuit 54 to the floppy disk driver 53, and saves this CD-ROM 35 based on this positional information to display 

memory content in the floppy disk together with the present this ma P ima S e ( ste P and also dls P Ia y s a P resent 

positional information. position on this map (step E4). Next, the CPU 55 accesses 

It should be noted that the video signal I/F circuit 52 15 a floppy disk set to the floppy disk driver 53 in order to check 

corresponds to a video capture for converting a video signal as t0 whether or not the photographed image data is recorded 

into a digital image, and the digital image from this video on the floppy disk in correspondence with positional infor- 

signal I/F circuit 52 is acquired in to the CPU 55. mation near the present position (step E5). As a result of this 

Apparently, the video signal I/F circuit 52 may be arranged check operation, if the photographed image data is stored, 

as a serial interface circuit such as the RS-232C by which the 20 then a guidance representation for inquiring whether or not 

image may be acquired from the digital camera. the image is displayed is made and the process operation is 

[Operations of GPS On- Vehicle Terminal Unit 51] brought to the key entry condition (step E7). FIG. 17E 

FIG. 15 is a flow chart for explaining operations of image represents this display condition. When the cursor is set to 
recording in the GPS on-vehicle terminal unit 51 of the third a position of "YES" (step E8), this image is displayed (step 
image processing system. FIG. 16 is a flow chart for 25 E9). In this case, the photographed image is displayed 
describing operation of the normal navigation mode in the instead of the map image. Alternatively, as previously 
GPS on-vehicle terminal unit 51. It should be noted that a explained, the map image is synthesized with the photo- 
program capable of realizing various functions defined in graphed image, so that the synthesized image may be 
these flow charts is previously stored in the above-explained displayed. 

storage medium in a program code format readable by the 30 In the GPS on-vehicle terminal unit 51 with the above- 
CPU 55. described arrangement, the arbitrary image data is acquired 

When the image record key is manipulated under such a from the video camera 61, and then the arbitrary image data 

condition that a map image of a present place (point) is can be recorded/saved in relation to the positional informa- 

displayed in the car navigation mode (see FIG. 17 A), a tion indicative of this photograph place. Furthermore, when 

guidance representation is made which inquires whether or 35 the vehicle again comes to this photograph place, the pho- 

not an image of the present place is recorded (see FIG. 17B). tographed image about this photograph place, which has 

In this case, a cursor is positioned to "YES" and also the previously been recorded/saved, can be displayed, 

video camera 61 is connected to this GPS on-vehicle ter- Accordingly, the car navigation function can be improved, 

minal unit 51. In other words, every time the positional information is 

As a result, the CPU 55 commences to execute the image 40 changed during the vehicle drive operation, the positional 

recording process described in the flow chart of FIG. 15. information recorded/saved in the floppy disk is retrieved. If 

First, a video signal derived from the video camera 61 is the photographed image corresponding to this positional 

converted into digital image data by the video signal I/F information is present, then this photographed image is 

circuit 52, and then this digital image data is acquired by the displayed. For instance, it is possible to drive your own 

CPU 55 (step Dl). In this case, the CPU 55 temporarily 45 vehicle while being guided by scenes which have previously 

stores this digital image data into the RAM 58 (step D2), and been photographed by a third party. The photographed 

displays the image data stored in this RAM 58 (step D3). At images acquired in this GPS on-vehicle terminal unit 51, 

this time, as shown in FIG. 17C, a guidance representation moreover, may be utilized not only for route guidance, but 

is made which indicates that a button should be depressed at also for advertisements of shops, or as a guide to tourist 

a scene to be recorded. Then, since the above-described 50 resorts. 

operations (defined from step Dl to step D3) are repeated As apparent from the foregoing descriptions, the present 
until an acquisition button is depressed (step D4), a moving invention is not limited only to the above-described first to 
picture is displayed on the display device 8. Now, when the third embodiments, but may be modified, changed, and 
acquisition button is depressed (step D4), positional infor- substituted without departing from the technical spirit scope 
mation for a file, which indicates this photograph place, is 55 of the present invention. For instance, although the photo- 
formed based upon the present positional information of the graphed images are recorded in correspondence with the 
car navigation. Then, this positional information is furnished positional information in the respective embodiments, the 
via the disk I/F circuit 54 to the floppy disk drive 53 so as map image data may be further stored in relation thereto, 
to be recorded/saved in the floppy disk (step D5). Then, the Alternatively, sound data entered from the video camera 
image data stored in the RAM 58 is converted into image 60 may be further stored in relation to the photographed image 
data having a format for a file, and the format-converted and the positional information. 

image data is recorded/saved in the floppy disk in corre- Not only the map image and the photographed image are 

spondence with the positional information (step D6). It displayed in the synthesized form, but also these images may 

should be understood that the file format conversion implies be displayed within one screen in a parallel manner, 

the above-described format conversion shown in FIG. 4, and 65 Moreover, these images may be printed out as hard copies, 

therefore the photographed image data is constituted by 240 Although the above-described embodiments have utilized 

dotsx240 dots. car navigation systems, the present invention may be simi- 
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larly applied to other types of navigation systems, e.g., a 
navigation system using the earth's magnetism instead of or 
in addition to the above-described satellite electromagnetic 
waves. 

What is claimed is: 

1. An image processing system comprising: 

registration designation means for designating a registra- 
tion of a photographed image; 

registration control means for, upon designation of the 
registration of the photographed image, acquiring from 
a GPS system image position data with respect to a 
position at which the photographed image was taken, 
and for registering the photographed image in corre- 
spondence with the acquired image position data; 

a map image display means for acquiring from the GPS 
system present position data with respect to a present 
position, for extracting a map image corresponding to 
the present position data, and for displaying the 
extracted map image on a display screen; 

determining means for determining whether or not there 
is a registered photographed image stored in correspon- 
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dence with image position data of a position near to the 
present position; and 
photographed image display means, responsive to a posi- 

5 tive determination by said determining means, for 
extracting the registered photographed image corre- 
sponding to the image position data of the position near 
to the present position, and displaying the extracted 
photographed image on the display screen. 

10 2. An image processing system according to claim 1, 
further comprising: 

message display means, also responsive to the positive 
determination by said determining means, for display- 
is ing a message indicating a presence of a nearby pho- 
tographed image; and 

wherein said photographed image display means com- 
prises means for displaying the nearby photographed 
20 image after the message is displayed. 

* * * * * 
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